Skip to content

feat(nucleus): Expose dataset_item_id on exported items, annotations and predictions - #473

Open
luke-e-schaefer wants to merge 1 commit into
masterfrom
lukeschaefer/export-dataset-item-id
Open

feat(nucleus): Expose dataset_item_id on exported items, annotations and predictions#473
luke-e-schaefer wants to merge 1 commit into
masterfrom
lukeschaefer/export-dataset-item-id

Conversation

@luke-e-schaefer

@luke-e-schaefer luke-e-schaefer commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Batch exports only ever returned reference_id on the exported item, so keying an exported prediction back to a Nucleus dataset item meant a second lookup.

This is an inconsistency fix, not a new field: the single-item endpoints (/dataset/:id/loc, /refloc, /iloc) already return dataset_item_id, and the public API docs (ApiDocsPage/slices/batch-export.md) already document it in the batch-export response. The batch export was the odd one out.

Pairs with scaleapi PR: luke/nucleus-export-dataset-item-id.

What changed

dataset_item_id now appears everywhere reference_id already does:

  • DatasetItem gains the field, wired through from_json — the single deserialization entry point, so every SDK path that returns an item picks it up, not just exports.
  • Every Annotation / Prediction subclass gains it (box, line, polygon, keypoints, cuboid, category, multicategory, segmentation). convert_export_payload stamps it down from the item exactly as it already does for reference_id.
  • The scene/video export docstring documents it on each track frame.

Read-only semantics

The field is server-assigned, and mirrors how DatasetItem.phash was done:

behaviour
from_json populated
locally constructed None
__eq__ excluded (compare=False)
to_payload absent — uploads unchanged

The __eq__ exclusion matters: without it, every existing test that compares a locally-built annotation against its round-tripped twin would start failing.

Exports from a backend that doesn't return the field leave it None rather than throwing, so this SDK version is safe against an un-upgraded backend.

Verification

The test suite requires live API keys (conftest.py hard-asserts on NUCLEUS_PYTEST_API_KEY), so it could not be run locally. Verified offline instead by driving convert_export_payload directly:

  • every geometry type on both the annotation and prediction paths carries the id
  • a local object still compares equal to its round-tripped self
  • to_payload output is unchanged
  • an item payload with no dataset_item_id yields None everywhere and does not raise

Formatted with the repo-pinned black 23.12.1 (a newer local black introduced unrelated reformatting, which was reverted).

🤖 Generated with Claude Code

Greptile Summary

The PR exposes the backend-assigned dataset_item_id on dataset items and all supported annotation and prediction objects while preserving upload payloads and existing equality behavior.

  • Adds optional, read-only dataset_item_id fields and deserialization across item, annotation, and prediction models.
  • Propagates each exported item's identifier to its converted annotations and predictions.
  • Documents scene-frame identifiers and releases the change as version 0.20.1.

Confidence Score: 5/5

The PR appears safe to merge with no concrete blocking or non-blocking defects identified.

The identifier is populated defensively from optional backend data, propagated consistently across supported export object types, excluded from equality, and omitted from outbound payloads.

Important Files Changed

Filename Overview
nucleus/annotation.py Adds an optional non-comparing dataset item identifier to every supported annotation model and deserializer without changing upload serialization.
nucleus/prediction.py Threads the optional identifier through all prediction constructors and deserializers while retaining inherited payload behavior.
nucleus/dataset_item.py Exposes the server-assigned identifier on deserialized dataset items without including it in uploads or equality.
nucleus/utils.py Copies the parent item identifier onto every annotation and prediction converted from batch-export payloads.
nucleus/dataset.py Updates scene/video export documentation to describe the identifier supplied on each frame.
pyproject.toml Advances the package version from 0.20.0 to 0.20.1.
CHANGELOG.md Documents the new exported identifier and its read-only, backward-compatible semantics.

Reviews (1): Last reviewed commit: "feat: expose dataset_item_id on exported..." | Re-trigger Greptile

…ctions

Batch exports only ever returned reference_id, so keying an exported
prediction back to a Nucleus dataset item meant a second lookup. The
single-item endpoints (/loc, /refloc, /iloc) already return
dataset_item_id, and the public batch-export docs already document it —
the batch export was the odd one out.

Adds the field everywhere reference_id already appears:

- DatasetItem gains dataset_item_id, wired through from_json (the single
  deserialization entry point, so every SDK path that returns an item
  picks it up).
- Every Annotation/Prediction subclass gains it, stamped down from the
  item by convert_export_payload exactly as reference_id already is.
- Scene/video exports carry it on each track frame.

It is server-assigned and read-only, mirroring DatasetItem.phash:
populated by from_json, None on locally-constructed objects, excluded
from __eq__ so a local object still compares equal to its round-tripped
self, and absent from to_payload so uploads are unchanged. A backend
that does not return it leaves the field None.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@luke-e-schaefer luke-e-schaefer changed the title [DE-XXXX] Expose dataset_item_id on exported items, annotations and predictions feat(nucleus): Expose dataset_item_id on exported items, annotations and predictions Aug 17, 2026
@luke-e-schaefer
luke-e-schaefer marked this pull request as ready for review August 17, 2026 17:04
@luke-e-schaefer luke-e-schaefer self-assigned this Aug 17, 2026
@luke-e-schaefer
luke-e-schaefer requested a review from a team August 17, 2026 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant